Traceability export - extremely slow

Hi all,

I wrote a quick script to export all testcases for each of our customers requirements. The scirpt works like it should, but my problem is, that it is extremely slow. It ran for 12 hours and still isn't finished. Could somebody take a look at it and tell me what is wrong? I would really appreciate it!
 

pragma encoding, "UTF-8"
pragma runLim,0 // Runtime limit; Deactivates Doors Timeout
pragma stack, 50000 // increase the default stack size from 16384 to 50000 due to a lot of recursion and stack overflow problems
 
/*
...
initialize constants...
...
*/
        Stream  Output                                                                  =       write sOUTPUTFILE
                string  sProgressedObjects                                              =       null
                Buffer  b                                                                               =       create()
                Buffer  bOutput                                                                 =       create()
                
string ReplaceSubstring(string InputString, ReplaceWith){ // Replaces " (quotation marks) in InputString with ReplaceWith
        int j
        int intLength 
        // b = create // create buffer
        setempty(b)
        intLength=length InputString 
 
        for(j=0;j<intLength;j++){ 
                if(InputString[j]=='\"') b+=ReplaceWith 
            else  b+=InputString[j]"" 
        }
        return b "" 
}             
                
void getTestcases(Module mStartModule, Object oStartObject) {
        Module          mTestModule                             =       null
        Object          oTestObject                             =       null
        Link            lTestLink                               =       null
        ModName_        srcModRef                               =       null
        string          sTestcaseText                   =       null
        string          sTestcaseASIL                   =       null
        string          sTestrunState                   =       null
        string          sAttributeTestrunState  =       null
        int                     iTestrunCount                   =       null
 
        for srcModRef in oStartObject <- sLINKMODULE_VERIFY do {
                if(!open(module(fullName(srcModRef)))) {
                        read(fullName(srcModRef), false)
                }
        }
 
        for lTestLink in oStartObject <- sLINKMODULE_VERIFY do {          
                oTestObject = source(lTestLink)
                mTestModule = module(oTestObject)
                
                iTestrunCount = mTestModule.sATTRIBUTE_T3_TEST_RUN_COUNT
                sAttributeTestrunState = sATTRIBUTE_TEST_RUN_STATE" "iTestrunCount""
                
                
                sTestcaseText = oTestObject."Object Text"
                        sTestcaseText = "\""ReplaceSubstring(sTestcaseText, "\"\"")"\"" // Add quotation marks and double the existing quotation marks
                sTestcaseASIL = oTestObject.sATTRIBUTE_TARGET_RELEASE_PRIORITY
                        sTestcaseASIL = "\""ReplaceSubstring(sTestcaseASIL, "\"\"")"\""
                sTestrunState = oTestObject.sAttributeTestrunState
                
                bOutput += ";;;"identifier(oTestObject)";"sTestcaseText";"sTestcaseASIL";"sTestrunState"\n"
        }
}
 
void getInLinkObjects(Module mStartModule, Object oStartObject) {
        Module          mCurrentModule          =       null
        Object          oCurrentObject          =       null
        Link            lCurrentInlink          =       null
        ModName_        srcModRef                       =       null
        string          sCurrentObject          =       null
        int                     iOffset                         =       null
        int                     iLength                         =       null
 
        for srcModRef in oStartObject <- "*" do {
                if(!open(module(fullName(srcModRef)))) {
                        read(fullName(srcModRef), false)
                }
        }
 
        for lCurrentInlink in oStartObject <- "*" do {
                
                oCurrentObject = source(lCurrentInlink)
                mCurrentModule = module(oCurrentObject)
                
                sCurrentObject = identifier(oCurrentObject)
                
                if (oStartObject == oCurrentObject) continue // prevent infinite loops
                if (findPlainText(sProgressedObjects, sCurrentObject, iOffset, iLength, false)) continue // prevent infinite loops and duplicate objects
                
                sProgressedObjects = sProgressedObjects sCurrentObject ";"
                
                getTestcases(mCurrentModule, oCurrentObject)
                getInLinkObjects(mCurrentModule, oCurrentObject)
        }
}
 
void startExport(DB x) {
        Module  mCustomerModule         =       current Module
        Object  oCustomerObject         =       null
        string  sCurrentCategory        =       null
        string  sCurrentAM_Status       =       null
        string  sCusReqText                     =       null
        string  sCusReqASIL                     =       null
        int     intProgressBarLimit     =       null
        int             intProgressBarCount     =       null
 
        for oCustomerObject in mCustomerModule do intProgressBarLimit++ // Count objects in mStartModule -> ProgressBar limit
        progressStart(x, "Traceability Matrix Export", "Starting", intProgressBarLimit) // initialise ProgressBar
        
        bOutput = "Customer Requirement ID;Customer Requirement Text;Customer Requirement ASIL;Testcase ID;Testcase Text;Testcase ASIL;Testrun State\n"
        
                for oCustomerObject in mCustomerModule do {
                        // ProgressBar Start
                                intProgressBarCount++
                                strMessage = "Module: " name(mCustomerModule) "\n" "Object: " intProgressBarCount "/" intProgressBarLimit " (" identifier oCustomerObject ")"
                                progressStep intProgressBarCount
                                progressMessage strMessage
                                if (progressCancelled) { // user input: Cancel
                                        if (confirm("Cancel traceability export?")) { // ask user to confirm abortion
                                                progressStop // stop progressbar
                                                halt // stop script
                                        }
                                }
                        // ProgressBar End
                
                        sCurrentCategory = oCustomerObject.sATTRIBUTE_CATEGORY
                        sCurrentAM_Status = oCustomerObject.sATTRIBUTE_AM_STATUS
                        
                        if (sCurrentCategory == sCATEGORY_VALUE_INPUT_REQUIREMENT && (sCurrentAM_Status == sAM_Status_VALUE_ACCEPTED || sCurrentAM_Status == sAM_Status_VALUE_ACCEPTED_RESTRICTION)) {
                                sCusReqText = oCustomerObject."Object Text"
                                        sCusReqText = "\""ReplaceSubstring(sCusReqText, "\"\"")"\""
                                sCusReqASIL = oCustomerObject.sATTRIBUTE_ASIL
                                        sCusReqASIL = "\""ReplaceSubstring(sCusReqASIL, "\"\"")"\""
                                
                                bOutput += identifier(oCustomerObject)";"sCusReqText";"sCusReqASIL"\n"
                                sProgressedObjects = null
                                getInLinkObjects(mCustomerModule, oCustomerObject)
                                
                                Output << stringOf(bOutput)
                                setempty(bOutput)
                        }
                }
        
        progressStop // stop ProgressBar
        
        close (Output) // Close stream
        
        delete b
        delete bOutput
        
        
        infoBox "Done"
        
        hide x
}
 
void main {
        DB progressDB = create("Export", styleCentered) // create Messagebox 
 
        label(progressDB, "Start traceability export?") // Messagebox -> start script?
        
        apply(progressDB, startExport) // user input: Apply
        
        realize progressDB
        setCenteredSize(progressDB, 320, 130)
 
        show progressDB
        
        halt
}

Many thanks in advance,

Johnny

Re: Traceability export - extremely slow
Johnny_P - Fri Oct 12 03:06:00 EDT 2012

Is it not possible to edit my previous post?
Anyways, I removed some lines, which I think, have nothing to do with my issue. So here is an update of the (reduced) code :)
 

pragma encoding, "UTF-8"
pragma runLim,0 // Runtime limit; Deactivates Doors Timeout
pragma stack, 50000 // increase the default stack size from 16384 to 50000 due to a lot of recursion and stack overflow problems
 
/*
...
initialize constants
...
*/
 
Stream    Output        =       write sOUTPUTFILE
string  sProgressedObjects      =       null
Buffer  bOutput =       create()
                
void getTestcases(Module mStartModule, Object oStartObject) { // add all linked testcase IDs to the buffer bOutput
Module  mTestModule     = null
Object  oTestObject     = null
Link    lTestLink =     null
ModName_ srcModRef =    null
        
        // open all "verify in-link" modules:
        for srcModRef in oStartObject <- sLINKMODULE_VERIFY do { 
                if(!open(module(fullName(srcModRef)))) {
                        read(fullName(srcModRef), false)
                }
        }
        
        for lTestLink in oStartObject <- sLINKMODULE_VERIFY do { // for each "verify - in-link" in the current object:   
                oTestObject = source(lTestLink)
                mTestModule = module(oTestObject)
 
                bOutput += ";;;"identifier(oTestObject)"\n" // add the current testcase ID to the buffer
        }
}
 
void getInLinkObjects(Module mStartModule, Object oStartObject) {
Module  mCurrentModule = null
Object  oCurrentObject = null
Link    lCurrentInlink = null
ModName_ srcModRef = null
string  sCurrentObject = null
int     iOffset = null
int     iLength = null
        
        // open all in link modules:
        for srcModRef in oStartObject <- "*" do {
                if(!open(module(fullName(srcModRef)))) {
                        read(fullName(srcModRef), false)
                }
        }
        
        for lCurrentInlink in oStartObject <- "*" do {
                
                oCurrentObject = source(lCurrentInlink)
                mCurrentModule = module(oCurrentObject)
                
                sCurrentObject = identifier(oCurrentObject)
                
                if (oStartObject == oCurrentObject) continue // prevent infinite loops
                if (findPlainText(sProgressedObjects, sCurrentObject, iOffset, iLength, false)) continue // prevent infinite loops and duplicate objects
                
                sProgressedObjects = sProgressedObjects sCurrentObject ";"
                
                getTestcases(mCurrentModule, oCurrentObject)
                getInLinkObjects(mCurrentModule, oCurrentObject)
        }
}
 
void startExport() {
Module  mCustomerModule = current Module
Object  oCustomerObject = null
 
        for oCustomerObject in mCustomerModule do {
                bOutput += identifier(oCustomerObject)"\n"
                sProgressedObjects = null
                getInLinkObjects(mCustomerModule, oCustomerObject)
                
                Output << stringOf(bOutput)
                setempty(bOutput)
        }
 
        
close (Output) // Close stream
 
delete bOutput
}
 
startExport()

Re: Traceability export - extremely slow
Peter_Albert - Fri Oct 12 04:21:51 EDT 2012

pragma encoding, "UTF-8"
pragma runLim,0 // Runtime limit; Deactivates Doors Timeout
pragma stack, 50000 // increase the default stack size from 16384 to 50000 due to a lot of recursion and stack overflow problems
 
/*
...
initialize constants
...
*/
 
Stream    Output        =       write sOUTPUTFILE
string  sProgressedObjects      =       null
Buffer  bOutput =       create()
                
void getTestcases(Module mStartModule, Object oStartObject) { // add all linked testcase IDs to the buffer bOutput
Module  mTestModule     = null
Object  oTestObject     = null
Link    lTestLink =     null
ModName_ srcModRef =    null
        
        // open all "verify in-link" modules:
        for srcModRef in oStartObject <- sLINKMODULE_VERIFY do { 
                if(!open(module(fullName(srcModRef)))) {
                        read(fullName(srcModRef), false)
                }
        }
        
        for lTestLink in oStartObject <- sLINKMODULE_VERIFY do { // for each "verify - in-link" in the current object:   
                oTestObject = source(lTestLink)
                mTestModule = module(oTestObject)
 
                bOutput += ";;;"identifier(oTestObject)"\n" // add the current testcase ID to the buffer
        }
}
 
void getInLinkObjects(Module mStartModule, Object oStartObject) {
Module  mCurrentModule = null
Object  oCurrentObject = null
Link    lCurrentInlink = null
ModName_ srcModRef = null
string  sCurrentObject = null
int     iOffset = null
int     iLength = null
        
        // open all in link modules:
        for srcModRef in oStartObject <- "*" do {
                if(!open(module(fullName(srcModRef)))) {
                        read(fullName(srcModRef), false)
                }
        }
        
        for lCurrentInlink in oStartObject <- "*" do {
                
                oCurrentObject = source(lCurrentInlink)
                mCurrentModule = module(oCurrentObject)
                
                sCurrentObject = identifier(oCurrentObject)
                
                if (oStartObject == oCurrentObject) continue // prevent infinite loops
                if (findPlainText(sProgressedObjects, sCurrentObject, iOffset, iLength, false)) continue // prevent infinite loops and duplicate objects
                
                sProgressedObjects = sProgressedObjects sCurrentObject ";"
                
                getTestcases(mCurrentModule, oCurrentObject)
                getInLinkObjects(mCurrentModule, oCurrentObject)
        }
}
 
void startExport() {
Module  mCustomerModule = current Module
Object  oCustomerObject = null
 
        for oCustomerObject in mCustomerModule do {
                bOutput += identifier(oCustomerObject)"\n"
                sProgressedObjects = null
                getInLinkObjects(mCustomerModule, oCustomerObject)
                
                Output << stringOf(bOutput)
                setempty(bOutput)
        }
 
        
close (Output) // Close stream
 
delete bOutput
}
 
startExport()

It might be your approach to "// prevent infinite loops and duplicate objects" using a string and 'findPlainText'. The string 'sProgressedObjects' becomes larger and larger, which means that 'findPlainText' becomes slower and slower.
I just wrote a test script which runs through the current Module a few times and does this test, and it is running for several minutes now and did not finish yet. In the same script, I also used an alternative approach using a global Buffer and 'contains' instead.
For this, you need a

Buffer bProgressedObjects = create(); bProgressedObjects = ""

sProgressedObjects = sProgressedObjects sCurrentObject ";"

becomes

bProgressedObjects += sCurrentObject; bProgressedObjects += ";"

and

if (findPlainText(sProgressedObjects, sCurrentObject, iOffset, iLength, false))

becomes

if (contains(bProgressedObjects, sCurrentObject, 0) > -1)

And finally,

delete bProgressedObjects

As the script is still running, I cannot tell you yet whether this is really faster, but for sure the string-based approach clearly causes the very slow execution. Probably you can come up with a more clever way to avoid infinite loops.

Regards,

Peter

Re: Traceability export - extremely slow
Johnny_P - Fri Oct 12 05:05:24 EDT 2012

Hi Peter,

in the meantime I made the following change:

I changed the "*" in the following rows to defined linkmodules. This way I was able to cut the run time down to about 30 minutes

for srcModRef in oStartObject <- "*" do {
//...
}
for lCurrentInlink in oStartObject <- "*" do {
//...
}

Now I applied your fix and the export is done in a couple of minutes!

I also applied it to my Traceability Matrix export script (because it had the same method of preventing infinite loops) and the speed also increased clearly!
Thank you very much for your help,
BR
Johnny